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SUMMARY 


OBJECTIVE 

The  purpose  of  this  research  was  to  investigate  decision  response  times  in  a  dynamic  tactical 
scenario  in  which  participants  interacted  with  a  virtual  command-post  environment.  Fifty-two 
Marines  with  varying  amounts  of  conunand-post  experience  assessed  the  situation  as  it  developed, 
determined  tactical  leverage  points,  formed  a  plan  of  action,  and  submitted  battle  orders.  Two 
scenarios,  which  differed  in  the  level  of  certainty  in  the  information  provided,  were  studied.  The 
tactical  decision  process  was  modeled  and  analyzed  in  the  following  sequential,  cognitive  stages: 
situation  assessment,  course  of  action  selection,  course  of  action  execution. 

RESULTS 

Results  show  that  the  time  required  to  assess  the  situation  was  significantly  different  between  the 
experience  groups  (p  <  .05),  revealing  that  the  High-Experience  group  took  considerably  longer  than 
the  Lx)w-Experience  group  to  assess  the  situation.  However,  once  the  assessment  was  complete,  the 
selection  of  a  course  of  action  (COA)  was  significantly  faster  for  the  High-Experience  group  than  the 
Low-Experience  group.  In  addition,  a  statistically  significant  main  effect  of  Task  Certainty  was 
found  indicating  that  COA  selection  under  conditions  of  Low  Certainty  took  significantly  longer  than 
under  conditions  of  High  Certainty.  Time  required  for  COA  execution  indicated  a  significant  main 
effect  of  Experience  (p  <  .05),  a  main  effect  of  Task  Certainty  approaching  statistical  significance 
(p  =  .067),  and  a  statistically  significant  interaction  (p  <  .05).  These  results  indicate  that  the  time 
needed  to  execute  the  COA,  once  determined,  is  significantly  less  for  the  highly  experienced  indi¬ 
viduals  under  conditions  of  low  certainty.  However,  under  conditions  of  high  certainty,  no  statisti¬ 
cally  significant  time  differences  were  found  distinguishing  the  High-  and  Low-Experience  groups. 
The  High-Experience  group  was  significantly  more  accurate  than  the  Low-Experience  group  for 
developing  an  appropriate  COA. 
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INTRODUCTION 


Timely  and  effective  decision-making  is  essential  to  the  success  of  any  military  operation.  It  is  for 
this  reason  that  the  Department  of  Defense  spends  considerable  time  and  resources  to  develop  the 
decision-making  processes  of  U.S.  military  personnel.  Two  general  models  have  been  used  to 
describe  the  decision-making  process:  (1)  Analytical  Decision-Making  (ADM)  and  (2)  Intuitive  or 
Naturalistic  Decision-Making  (NDM). 

ADM  has  a  longer  history  than  NDM  and  has  been  studied  more  thoroughly  in  the  laboratory  set¬ 
ting.  Historically,  this  method  of  decision-making  has  been  explained  in  terms  of  probability  of  a 
particular  outcome.  This  method  generally  relies  on  a  logical  analysis  of  the  situation  upon  which  the 
decision  is  based  and  thus  uses  analytical  or  computational  procedures.  This  approach  requires  the 
individual  decision-maker  to  evaluate  all  options,  weigh  the  cost  and  benefits  of  each,  and  identify  a 
course  of  action  that  will  lead  to  the  best-expected  outcome  (Bergstrand,  1997).  This  has  been  the 
traditional  method  used  by  the  military  in  training  decision-makers  to  collect  and  analyze  information 
and  to  then  generate  a  variety  of  candidate  solutions  to  the  tactical  problem.  Each  of  the  solutions  is 
evaluated  based  upon  pre-established  criteria.  The  solution  with  the  best  fit  is  then  selected  as  the 
unit’s  course  of  action,  and  appropriate  orders  are  then  issued.  This  method  can  provide  excellent 
results  given  reliable  information,  well-understood  selection  criteria,  a  clearly  defined  objective,  and 
plenty  of  time.  Nonetheless,  there  are  clear  disadvantages  to  this  method,  especially  for  the  military. 
In  combat  or  tactical  situations,  information  is  often  unreliable  and  inaccurate,  leading  to  situation 
ambiguity  or  uncertainty.  Rarely  is  there  time  to  complete  the  time-consuming  decision-making 
process  presumed  by  the  ADM  method.  The  high  uncertainty  found  in  real-world  situations  raises 
questions  about  the  utility  of  ADM  for  applied  tasks,  which  has  led  to  many  studies  attempting  to 
find  an  appropriate  decision-making  model  for  the  natural  environment. 

Shafir  (1994)  indicated  that,  in  real-world  decision-making,  experts  do  not  have  (nor  do  they  take) 
the  time  to  assess  all  possible  alternatives.  Instead,  they  restrict  their  processing  to  an  ideal  set  of 
options  for  further  review.  Other  studies  have  repeatedly  demonstrated  that  the  classical  model 
involving  purely  mathematical  formulations  of  probabilistic  or  rational  assessment  do  not  capture 
the  complexities  of  human  decision-making  in  the  field  (Jungermann,  1997;  Kahneman,  Slovic,  and 
Tversky,  1985).  Although  the  process  of  decision-making  has  been  investigated  for  centuries,  only  in 
the  last  few  decades  have  investigators  systematically  addressed  how  decisions  are  made  in  a 
dynamic,  real-world  environment. 

Decision-making  strategies  in  natural  environments  appear  to  be  very  different  from  those  found 
in  static  laboratory  settings.  Recent  work  in  naturalistic  decision-making  focuses  on  identifying  the 
sources  of  uncertainty  and  how  people  cope  with,  or  manage,  uncertainty  (Klein,  Schmitt,  McCloskey, 
Heaton,  Klinger,  and  Wolf,  1996;  Cohen,  Freeman,  and  Wolf,  1996;  Lipshitz  and  Strauss,  1997; 
Yaniv  and  Foster,  1995;  Zsambok  and  Klein,  1997).  Zsambok  (1997,  p.  4)  defines  naturalistic 
decision-making  as  “...  the  way  people  use  their  experience  to  make  decisions  in  the  field.”  The 
NDM  method  tends  to  rely  on  simulations  to  elicit  information  from  experts  in  a  particular  domain 
and  highlights  the  differences  in  decision-making  between  experts  and  novices.  For  example,  Stokes, 
Kemper,  and  Kite  (1997)  used  a  computer-displayed  instrument  panel  to  study  in-flight  decision¬ 
making  of  expert  and  novice  pilots.  The  pilots  responded  to  emergencies  by  indicating  their  actions 
and  rationale.  Expert  pilots  responded  more  quickly,  generated  more  alternative  actions,  and  identi¬ 
fied  more  relevant  cues  than  did  novice  pilots.  Novices,  on  the  other  hand,  tended  to  be  biased 
toward  their  first  (and  usually  only)  option. 
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A  bias  toward  the  first-derived  solution  is  a  strategy  that  is  often  seen  in  uncertain  situations 
(Klein,  Schmitt,  McCloskey,  Heaton,  and  Wolf,  1996).  Other  common  strategies  for  coping  with 
uncertainty  include  an  over-reliance  on  past  experience  and  positive  outcomes  (Cohen  and  Wallsten, 
1992),  minimization  of  negative  evidence  (Reece  and  Matthews,  1993),  reasoning  based  upon  the 
subjective  weighing  of  pros  and  cons  (Lipshitz  and  Strauss,  1996),  and  delaying  or  deferring  deci¬ 
sions  (Arai,  1997;  Dhar,  1996, 1997).  For  example,  Dhar  (1997)  evaluated  consumer  preferences  for 
competing  alternatives  and  found  that  uncertainty  (created  when  no  single  alternative  had  an  advan¬ 
tage  over  other  alternatives)  led  research  subjects  to  defer  a  choice. 

Military  studies  have  reported  that  the  ADM  procedures  called  for  in  doctrine  and  training  are 
rarely  practiced  in  the  field.  The  doctrinal  process  of  generating  and  evaluating  three  courses  of 
action  leads  to  tactical  inefficiencies  when  uncertainty  is  high  and  when  time  is  short  (Bergstrand, 
1997).  Bergstrand  has  identified  a  list  of  features  (table  1)  describing  a  naturalistic  decision-making 
environment.  The  resemblance  between  the  features  of  an  NDM  environment  and  a  military 
decision-making  environment  is  extraordinary. 

Table  1 .  Features  of  naturalistic  decision-making. 

•  Ill-structured  problems _ 

•  Uncertain,  dynamic  environments 

•  Shifting,  ill-defined  or  competing  goals 

•  Action-feedback  loops 

•  Time  stress 

•  High  stakes 

•  Experienced  decision-makers _ 

•  Multiple  players 

•  Organizational  goals  and  norms 


Several  models  of  decision-making  in  the  natural  enviromnent  have  been  suggested  (Klein,  1993). 
The  model  that  has  been  specifically  identified  by  Klein  as  the  most  applicable  for  the  military  is 
Recognition-Primed  Decision-Making  (RPD).  This  model  highlights  the  importance  of  intuitive 
situation  assessment  as  the  basis  for  effective  decision-making.  The  general  premise  of  this  model  is 
that,  in  an  operational  environment,  people  rarely  weigh  alternatives  and  compare  them  in  terms  of 
expected  value  or  utility  (Lehto,  1997).  Rather,  experienced  decision-makers  try  to  recognize  familiar 
patterns,  features,  or  prototypes.  Potential  solutions  are  generated  sequentially  and  evaluated  men¬ 
tally  to  determine  if  the  solution  is  workable.  If  the  solution  is  not  workable,  it  is  modified  or  elimi¬ 
nated.  The  process  continues  until  a  workable  solution  is  generated  and  selected. 

Table  2  lists  the  key  features  of  the  Recognition-Primed  Decision-Making  model  as  outlined  by 
Bergstrand  (1997).  The  key  to  this  model  is  level  of  expertise,  in  the  specific  content  area,  of  the 
individual  making  the  decision.  As  Lehto  (1997)  points  out,  “the  most  general  conclusion  that  can  be 
drawn  from  this  area  of  research  is  that  people  use  different  decision  strategies  depending  upon  their 
experience,  the  task,  and  the  decision  context”  (p.  1235). 
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Table  2.  Key  features  of  Recognition-Primed  Decision-Making  model. 


•  First  option  usually  workable,  NOT  random  generation  and  selective  retention  options 

•  Serial  generation  and  evaluation  of  options,  NOT  concurrent  evaluation 

•  Satisfying,  NOT  optimizing 

•  Evaluation  through  mental  simulation,  NOT  multi-attribute  utility  analysis,  decision  analysis, 
or  Bayesian  analysis 

•  Focus  on  elaborating  and  improving  options,  NOT  choosing  between  options 

•  Focus  on  situation  assessment,  NOT  decision  events 

•  Decision-maker  primed  to  act,  NOT  wait  for  complete  analysis 


Table  1  indicates  that  uncertainty  is  one  of  the  prime  attributes  of  the  NDM  model.  There  is 
little  question  that  decision-making  under  conditions  of  uncertainty  is  a  regular  feature  of  military 
decision-making.  Commanders  are  typically  required  to  make  decisions  and  develop  battle  plans 
without  the  benefit  of  knowing  the  intent  of  the  adversary,  how  large  or  well-equipped  he  is,  or 
where  he  may  be  at  any  given  point  in  time.  The  hterature  is  rich  with  reports  investigating  decision¬ 
making  under  uncertainty  (e.g.,  Rowe,  1994;  Lipshitz  and  Strauss,  1997)  and  studies  investigating 
situation  assessment  and  awareness  in  a  variety  of  environments  (Gaba,  Howard,  and  Small,  1995; 
Endsley,  1995;  O’Hare,  1992;  Salas,  Prince,  Baker,  and  Shrestha,  1995).  These  studies  and  others 
suggest  not  only  an  assortment  of  decision-making  strategies,  but  also  highlight  the  vital  role  that 
human  factors  engineers  can  play  by  designing  display  interfaces  that  better  support  dynamic 
decision-making  (Lehto,  1997). 

In  1998,  a  workshop  was  hosted  by  SSC  San  Diego  and  Pacific  Science  and  Engineering  Group 
with  the  goal  of  bringing  together  a  number  of  human  factors  researchers  to  speculate  on  new  ideas 
for  improving  command-post  situation  awareness  and  the  portrayal  and  handling  of  uncertain  infor¬ 
mation.  The  workshop  specifically  addressed  how  to  portray  uncertain  information,  such  as  the  intent 
and  positions  of  enemy  forces,  and  how  to  improve  situation  awareness  and  decision-making  under 
uncertainty. 

Human  factors  researchers  have  also  investigated  how  best  to  design  displays  to  optimally  present 
relevant  information  and  enhance  situation  awareness  and  decision-making  under  uncertainty.  St 
John,  Callan,  Proctor,  and  Holste  (2000)  examined  how  different  representations  of  uncertainty 
(graphic  vs.  text  based,  discreet  vs.  continuous)  concerning  enemy  intent,  composition,  and  position 
in  a  ground  war  battlefield  situation  affected  situation  awareness  in  terms  of  situation  assessment  and 
time  needed  to  make  a  decision.  They  found  that  Marines  with  relatively  little  command-post  experi¬ 
ence  were  more  likely  to  “wait-and-see”  prior  to  deciding  on  a  battle  plan  than  were  Marines  with 
more  experience  when  confironted  with  an  uncertain  tactical  scenario.  This  finding  is  in  concert  with 
the  results  of  Dhar  (1997),  indicating  that  the  likelihood  of  delaying  a  choice  is  contingent  upon  the 
ease  of  making  the  decision.  Individuals  with  little  experience  find  the  required  task  more  difficult 
than  an  individual  who  has  “been  there — done  that.”  However,  the  tactical  scenario  used  in  the  study 
by  St.  John  et  al.  was  static;  participants  were  provided  with  a  narrative  of  the  current  situation  along 
with  a  map  depicting  the  scenario  and  forced  to  choose  whether  they  would  make  a  decision  to  “act” 
or  to  “wait  for  additional  information.”  Choosing  to  wait  is,  of  course,  contrary  to  Marine  Corps 
doctrine,  which  is  to  act  as  quickly  as  possible. 

Decision-making  under  static  euid  dynamic  conditions  requires  different  decision-making  proc¬ 
esses,  and  the  differences  in  processing  strategies  noted  between  static  and  dynamic  environments 
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raise  several  questions.  Does  the  tactical  decision-making  strategy  used  by  expert  and  novice 
Marines  differ  between  static  and  dynamic  settings?  What  model  can  be  used  to  best  describe  how 
U.S.  Marines  make  tactical  decisions  in  a  dynamic  environment?  Would  the  time-stress  imposed  by 
the  dynamic  nature  of  the  task  spur  the  participants  toward  action  and  force  them  to  rely  on  their 
experience  and  intuition  to  determine  a  response,  thereby  accelerating  decision  cycles? 

This  study  investigated  decision  response  times  in  a  dynamic  tactical  scenario  in  which  partici¬ 
pants  were  immersed  in  a  virtual  command  post  and  were  asked  to  formulate  a  battle  plan  as  soon  as 
possible  pertaining  to  the  developing  tactical  situation.  Two  scenarios,  differing  in  the  level  of  cer¬ 
tainty  in  the  information  provided,  were  presented  to  determine  if  the  time  required  for  making  a  tac¬ 
tical  decision  was  related  to  the  level  of  information  certainty.  In  addition,  the  effect  of  experience 
was  evaluated  to  determine  how  expertise  affects  the  decision-making  process  in  a  dynamic  envi¬ 
ronment. 
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METHOD 


PARTICIPANTS 

Fifty-two  Marine  Corps  officers  and  non-conunissioned  officers  (NCOs)  with  varying  amounts  of 
command-post  experience  participated  in  the  experiment.  The  task,  a  dynamic  tactical  scenario,  was 
a  computer-based  simulation  of  command-post  operations  during  a  nighttime  land  warfare  engage¬ 
ment.  Participants  were  selected  from  the  Fifth  Marine  Regiment,  First  Marine  Division  at  Camp  . 
Pendleton,  CA.  Table  3  displays  the  number  and  ranks  of  participants.  Participants’  command-post 
experience  ranged  from  0  to  10  years  (M  =  477  days). 

Two  groups  were  formed  based  on  experience  in  the  command-post  environment.  A  median 
split  was  effected,  based  on  the  experience  level  of  the  participants.  The  Low-Experience  group 
consisted  of  29  participants,  ranging  in  experience  from  0  to  90  days  (M  =  35.17  days,  ^  = 

31 .62).  Fourteen  individuals  from  the  Low-Experience  group  participated  in  the  Low-Certainty 
condition  with  the  remainder  (15  individuals)  participating  in  the  High-Certainty  condition.  The 
High-Experience  group  consisted  of  23  participants  and  ranged  in  experience  from  91  to  3650 
days  (M  =  1034.96  days,  ^  =  991.52).  Thirteen  individuals  from  the  High- Experience  group 
participated  in  the  Low-Certainty  condition  with  the  remainder  (10  individuals)  participating  in 
the  High-Certainty  condition. 


Table  3.  Number  of  participants  indicated  by  rank. 


Rank 

Number  of  Participants 

•  Lieutenant  Colonel 

2 

•  Major 

6 

•  Captain 

6 

•  1st  Lieutenant 

13 

•  2nd  Lieutenant 

2 

•  Chief  Warrant  Officer 

4 

•  Master  Gunnery  Sergeant 

1 

•  Master  Sergeant 

2 

•  Gunnery  Sergeant 

2 

•  Staff  Sergeant 

9 

•  Sergeant 

5 

Twenty-seven  subjects  participated  in  the  Low-Certainty  condition,  and  the  remaining  25  subjects 
participated  in  the  High-Certainty  condition. 

MATERIALS 

The  experiment  involved  a  computer-controlled  scenario  adapted  from  a  tactical  decision  game 
published  in  Mastering  Tactics  (Schmitt,  1994),  written  in  Macromedia  Director  (version  7.0)  and 
executed  on  a  PC  laptop.  The  scenario  was  a  simulation  of  a  command  post  in  which  all  information 
was  presented  on  the  laptop  screen  and  over  headphones.  The  computer  display  was  a  composite  of 
information  available  within  a  field  command  post,  provided  on  a  single  display.  The  interface  was 
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iteratively  designed  after  conducting  interviews  with  a  large  number  of  Marines  with  varied  back¬ 
grounds  to  determine  the  type  of  information  required  and  the  optimal  way  to  display  that  informa¬ 
tion.  Figure  1  depicts  the  display  at  the  onset  of  the  task. 


You  rjre  1h<i  ccinirnonding  officer  of  2d  Bratlalion.  Sth  Marinos.  Your  i:cin5:ists  of Iwo  riflo  kompanios  on  fool,  one  rifie 

company  on  injo^rs.  u  '.■.■ufipcms  company,  a  tanlf  company  i;mlnu3;i.  a  TCi'iV  s&ctlan  on  HMM'iV'/s.  and  a  ETA.  Plaloon.  The 
Dragon?  r3nd  hoa^-.-y  rnaohino  gun?  of  your  weapons  company  have  been  attached  otd  1o  ihe  rifle  cornpanie?. 

You  wore  briefed  at  20(10.  Tomorrov/  morning  at  0100  Ihe  dr-nsion  begin?  a  major  offen?rv-e  south  acros?  the  river,  and  your 
baltallcin  will  sipe-arhe-ad  thfr  atlacK.  Your  orders:  are  lo  occupy/ lMe  esie-mPi'/  are-a  easil  oT  Durtiln  In  preparation  Tor  tho  eahy 
morning  offen?rve.  The  Ia1e?1  iniel  repoii?  (received  24  hour?  prion  indicate  that  friendly  force?  hold  Ihe  bridge  and  Ihe  rrverline 
In  tho  vlclniry  oT1he  Pridgo.  You  aro  movlnq  soulh  lowards  Ihe  asaomPt/  aroe  and  have  dispatched  2.  squads  ftom  me  ETA 
plaloon  lo  reconnoiter  the  routes  lo  Ihe  southv^e?!.  ?oidh.  and  southeast.  Squads  are  to  report  every  15  minutes.  AddHional 
rfrcdtinalssanto  assets  from  Regimental  HO  are  oporaling  south  orthfr  hvor.  HO  Is  on  the  move  and  will  be  oul  of  radio 


Figure  1 .  The  dynamic  display  at  the  onset  of  the  scenario. 

The  display  consisted  of  a  color  topographic  map  depicting  a  tactical  situation  with  associated  map 
manipulation  and  information  tools.  Tools  and  functions  available  to  the  participants  are  shown  in 
table  4. 

In  addition,  the  audio  channel  consisted  of  radio  traffic,  together  with  background  verbaliza¬ 
tions  and  ambient  background  noise,  to  simulate  the  command-post  environment.  Participants 
used  a  mouse  and  keyboard  to  interact  with  the  scenario. 
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Table  4.  Display  tools  and  functions  accompanying  the  computerized  map. 


Tool/Function 

Capability 

Panning 

Allows  the  map  to  be  scrolled  in  N/S  or  EAA/  directions 

Zooming 

Continuous  zoom-in  or  zoom-out  on  a  selected  area  of  the  map 

Grid  lines 

1-km  by  1-km  grid  overlay  for  the  map 

Unit  movement  histories 

Map  overlay  indicating  the  movement  history  of  friendly  units 

Axis  of  advance 

Map  overlay  indicating  unit  axis  of  advance 

Possible  enemy  location 

Map  overlay  indicating  potential  enemy  location  over  time 

Clock 

Depicts  scenario  time  (faster  than  real  time) 

Messages/archive 

Text  of  audio  recon  reports  (including  all  previous  reports) 

Review  orders 

Review  of  initial  brief  and  mission  orders 

Submit  orders 

Opens  a  text  window  in  which  to  write  and  submit  orders 

Assembly  area 

Indicates  location  of  assembly  area  on  the  map 

Tutorial 

A  tutorial  was  prepared  to  provide  a  brief  introduction  to  display  functionality.  The  tutorial  was  a 
noninteractive,  computer-based  training  tour  through  the  functionality  and  use  of  the  interface  and 
map  tools.  Participants  viewed  a  portion  of  the  scenario  map  and  received  descriptions  of  each  of  the 
interface  components  and  map  tools  while  audio  instructions  were  presented  through  the  headphones. 

The  tutorial  lasted  approximately  3  minutes  and  was  followed  by  a  brief  question-and-answer 
(Q&A)  period  to  provide  further  explanation,  if  required,  of  any  of  the  display  tools  and  functions. 
The  testing  scenario  did  not  begin  until  the  participant  was  ready  to  continue. 

Figure  2  shows  an  example  tutorial  display  screen  with  text  from  accompanying  audio  instructions. 
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“Periodically,  messages  will  come  in.  These  messages  will  be  listed  to  the 
left  in  the  message  area.  To  list  current  messages,  click  on  the  name  of  the 
troop  you  wish  to  address.  Then,  click  on  the  time  that  the  message 
arrived.  ” 

Figure  2.  The  tutorial  display  screen  describing  incoming  status  updates 
and  accompanying  audio  instructions. 

Test  Scenario 

The  test  scenario  depicted  a  land  warfare  engagement  and  battle  in  the  vicinity  of  the  fictional 
town  of  Durbin.  The  scenario  began  with  fiiendly  forces  planning  a  major  assault  south  across  the 
bridge,  scheduled  for  0100.  Participants  played  the  role  of  the  Commanding  Officer  of  a  U.S.  Marine 
Corps  battalion  whose  orders  were  to  occupy  an  assembly  area  east  of  the  town  of  Durbin  in  prepa¬ 
ration  for  spearheading  the  major  offensive  to  the  south.  Their  task  was  to  assess  the  situation  as  it 
developed,  determine  tactical  leverage  points,  develop  a  course  of  action,  and  submit  battle  orders. 
Reconnaissance  (Recon)  teams  were  sent  out  and  periodically  reported  back  to  the  command  post 
with  updated  information. 

Approximately  half  of  the  participants  entered  the  scenario  at  time  2000  (i.e.,  8:00  p.m.).  Intelli¬ 
gence  information  was  somewhat  dated,  and  the  recently  released  recon  teams  had  not  yet  reported. 
Initial  situation  ambiguity  was  high  for  this  High-Uncertainty  scenario.  The  remainder  of  the  par¬ 
ticipants  began  the  scenario  at  time  2130  (i.e.,  9:30  p.m.).  This  Low-Uncertainty  condition  provided 
more  detailed  and  recent  information  regarding  intelligence  updates  and  recon  reports,  and  the  initial 
situation  ambiguity  was  significantly  lower.  Participants  who  started  the  scenario  at  time  2000  had 
more  time  to  assess  the  situation  and  familiarize  themselves  with  the  battle  theater  prior  to  the  onset 
of  unexpected  events  (i.e.,  the  enemy  was  detected  in  a  location  that  was  thought  to  be  held  by 
friendly  forces).  Participants  who  started  the  scenario  at  time  2130  were  quickly  placed  in  a  situation 
where  the  enemy  was  in  unexpected  locations,  and  the  original  mission  orders  were  rapidly  becoming 
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obsolete.  In  using  the  different  conditions,  we  hoped  to  determine  whether  participants  would  react 
differently  given  the  additional  uncertainty  imposed  in  the  High-Uncertainty  scenario. 

PROCEDURE 

The  participants  were  run  individually  using  a  laptop  computer  with  headphones.  Figure  3  shows 
the  experimental  setup. 

The  participants  were  randomly  assigned  to  the  Low-  or  High-Uncertainty  condition.  Prior  to 
beginning  the  experiment,  participants  were  shown  the  tutorial.  Once  ready,  they  started  the  dynamic 
part  of  the  experiment  by  clicking  on  an  on-screen  start  button. 

Participants  were  instructed  to  select  a  tactical  leverage  point  from  a  list  of  geographical  locations 
as  soon  as  possible.  They  were  allowed  to  change  the  leverage  point  as  often  as  desired,  at  any  time 
during  the  scenario.  Participants  were  also  instructed  to  open  the  “Submit  Orders  Window”  only 
when  they  had  selected  their  course  of  action  (CO A)  committing  the  entire  battalion.  They  were  fur¬ 
ther  instructed  to  provide  battle-plan  orders  as  “Frag”  (fragmentary)  orders  rather  than  in  the  con¬ 
ventional  five-paragraph  format.  A  “Frag”  order  includes  the  commander’s  orders  and  task 
assignments  in  an  abbreviated  format. 


Figure  3.  The  experimental  setup. 

All  participant  interactions  with  the  display  using  keyboard  and  mouse  were  recorded  and  stored  in 
a  database  on  the  computer.  Additionally,  any  notes  taken  by  the  participants  were  collected  at  the 
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end  of  the  experiment.  Participants  completed  a  two-page  exit  questionnaire  after  the  experiment. 
Questions  centered  on  subjective  assessments  of  their  confidence  in  their  battle  plans,  the  effect  that 
the  situation  uncertainty  had  on  formulating  the  battle  plans,  and  the  utility  of  the  map  manipulation 
tools  for  the  task.  In  addition,  they  described  how  usefiil  they  thought  the  display  and  tools  would  be 
in  an  actual  command  post. 

Figure  4  displays  a  hypothetical  time  line  for  a  typical  participant  in  the  experiment.  Each  partici¬ 
pant  followed  the  same  sequence  of  events.  However,  the  number  of  leverage  points  selected  and  the 
timing  between  events  differed  between  subjects.  A  more  detailed  description  of  the  task  and  its 
relationship  with  NDM  is  discussed  in  the  next  section. 


START  Q&A  START 
TUTORIAL  PERIOD  SCENARIO 
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Figure  4.  A  hypothetical  time  line  for  a  typical  participant. 


DATA  ANALYSES 

In  concert  with  current  theory  on  Naturalistic  Decision-Making,  the  dependent  variables  collected 
were  used  to  compute  measures  related  to  the  model  of  Recognition-Primed  Decision-Making.  An 
additional  measure  (COA  Execution)  was  also  developed  and  is  described  in  more  detail  in  this  sec¬ 
tion. 

Figure  5  depicts  an  expanded,  annotated  time  line  reflecting  the  actions  of  a  hypothetical  partici¬ 
pant.  The  recording  of  time  began  at  point  A  for  all  participants  once  the  scenario  was  initiated. 
During  the  scenario,  participants  selected  the  geographical  location  that  they  considered  to  be  of  key 
tactical  importance.  This  was  referred  to  as  the  tactical  “leverage  point.”  They  were  allowed  to 
change  their  leverage-point  seleetion  at  any  time  during  the  scenario,  based  on  a  subjective  analysis 
of  the  information  they  were  receiving  throughout  the  scenario.  The  time  period  between  the  start  of 
the  task  and  the  final  leverage  point  (B)  was  considered  to  be  the  time  required  by  each  participant  to 
achieve  a  reasonable  understanding  of  the  tactical  situation  within  the  time  constraints  of  the  sce¬ 
nario.  This  time  period  also  reflects  a  period  of  recognition  priming,  leading  to  achievement  of  situa¬ 
tion  awareness. 

Once  the  participant  decided  on  a  final  leverage  point,  we  hypothesized  that  the  decision-making 
process  would  proceed  to  the  next  stage  of  processing,  which  is  to  decide  on  a  course  of  action.  COA 
Selection  was  operationally  defined  as  the  time  period  between  the  selection  of  the  final  leverage 
point  (B)  and  the  selection  of  the  Open  Orders  Window  (C),  the  first  step  required  to  initiate  a  battle 
plan. 
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The  validity  of  COA  Selection  as  an  accurate  measure  of  the  time  actually  required  to  select  the 
desired  course  of  action  could  be  questioned  due  to  the  dynamic  nature  of  the  task.  It  could  be  argued 
that  participants  may  have  selected  the  option  to  submit  orders  with  one  COA  in  mind,  but  then 
altered  their  plan  based  on  information  they  received  after  selecting  the  Open  Orders  Window  option. 
Therefore,  an  additional  measure  was  developed  that  captures  the  full  time  demands  of  COA  Selec¬ 
tion  but  also  includes  the  time  needed  to  execute  a  response.  This  time  period  was  referred  to  as  COA 
Execution  and  was  operationally  defined  as  the  time  period  between  the  selection  of  the  final  lever¬ 
age  point  (B)  and  the  submission  of  orders  (D). 
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RESULTS 


Data  were  collected  on  seven  dependent  measures  of  performance: 

1.  Time  of  first  leverage  point 

2.  Time  of  final  leverage  point 

3.  Number  of  leverage  points  selected 

4.  Accuracy  of  final  leverage  point 

5.  Time  to  submit  orders 

6.  Accuracy  of  course  of  action  selected 

7.  Total  time  on  task 

All  times  measures  were  calculated  using  the  initial  start  of  the  task  as  a  time  of  zero.  Independent 
variables  of  interest  were  Uncertainty  Level  (High,  Low)  and  Level  of  Command-Post  Experience 
(High,  Low).  Criteria  for  accuracy  were  provided  in  the  solution  section  of  Mastering  Tactics 
(Schmitt,  1994).  In  addition,  data  were  collected  regarding  the  subjective  rating  of  confidence  in  the 
battle  plan  selected  and  the  effect  that  uncertainty  had  upon  participants’  decision-making. 

Independent  two-factor  (Certainty  X  Experience)  Analyses  of  Variance  (ANOVAs)  were 
completed  for  each  of  the  dependent  variables. 

TIME  TO  FIRST  LEVERAGE  POINT 

A  two-way  ANOVA  revealed  that  neither  of  the  independent  variables  nor  the  interaction  were 
statistically  significant  at  the  .05  level  for  time  to  first  leverage  point.  However,  the  difference 
between  High-  and  Low-Experience  levels  was  suggestive  and  approached  statistical  significance 
(p  =  .067).  The  Low-Experience  group  (M  =  3.4  min)  tended  to  select  the  first  leverage  point  almost 
twice  as  quickly  as  their  more  experienced  (M  =  6.5  min)  counterparts. 

TIME  TO  LAST  LEVERAGE  POINT 

The  results  indicated  a  significant  effect  for  Experience  and  a  borderline  effect  for  Certainty,  but 
no  significant  interaction.  Low-Uncertainty  participants  (M  =  7.6  min)  tended  to  select  the  last 
leverage  point  faster  than  High-Uncertainty  participants  (M  =  12.3  min)  (p  =  .055). 

The  effect  for  Experience  was  statistically  significant  (F(l,43)  =  4.25,  p  <  .05),  with  Low- 
Experience  Marines  tending  to  select  the  final  leverage  point  faster  (M  =  7.7  min)  than  Marines 
with  more  experience  in  the  command  post  (M  =  12.6  min). 

NUMBER  OF  LEVERAGE  POINTS 

There  were  no  statistically  significant  effects  for  Certainty  or  Experience  in  terms  of  the  number  of 
leverage  points  selected. 
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TIME  TO  INITIATE  ORDERS 

The  overall  time  to  make  the  decision  to  act  (Open  Orders  Window)  was  significantly  different 
between  the  two  Certainty  groups  ®  (1,47)  =  23.1,  p  <  .001).  The  Low-Uncertainty  group  (M  =  7.84 
min)  made  the  decision  to  submit  orders  significantly  faster  than  the  High-Uncertainty  group 
(M  =  18.27  min).  There  was  no  statistically  significant  difference  based  upon  command-post  experi¬ 
ence  and  the  interaction  likewise  was  not  significant. 

TOTAL  TIME  ON  TASK 

Total  time  required  to  finish  the  task  was  found  to  be  significantly  different  between  Certainty 
groups  (F  (1,48)  =  19.14,  g  <  .001).  The  High-Uncertainty  group  spent  significantly  more  time 
(M  =  25.1 1  min)  completing  the  task  than  did  the  Low-Uncertainty  group  (M  =  15.36  min.).  Level  of 
command-post  experience  had  no  statistically  significant  effect  upon  time  to  complete  the  task  and 
no  significant  interaction  was  found. 

RANK 

Post  hoc  analyses  were  also  conducted  involving  military  rank.  Officers  selected  significantly 
fewer  leverage  points  (M  =  2.93  selections)  than  enlisted  participants  (M  =  5.47  selections)  regard¬ 
less  of  the  condition  in  which  they  participated  (F  (1,43)  =  8.08,  p  <  .005).  No  other  result  was 
statistically  significant. 

SITUATION  AWARENESS 

Figure  6  displays  the  mean  response  time  results  for  situation  awareness  acquisition  between 
groups  (High/Lx>w  Experience)  during  both  the  High-  and  Low-Uncertainty  conditions.  Acquisition 
of  situation  awareness  was  operationally  defined  as  the  time  elapsed  from  the  beginning  of  the 
experiment  until  the  selection  of  the  last  (or  only)  leverage  point.  Independent  analyses  of  each  main 
effect  revealed  a  statistically  significant  effect  for  Experience  (F(l,43)  =  4.25,  p  <  .05).  These  results 
indicate  that  the  High-Experience  group  spent  significantly  more  time  (M  =  12.6  min)  than  the  Low- 
Experience  group  (M  =  7.7  min)  analyzing  the  situation  in  the  development  of  situation  awareness. 
The  main  effect  for  task  Certainty  approached  statistical  significance  (p  =  .055)  as  the  time  taken  to 
acquire  awareness  under  High  Uncertainty  tended  to  be  greater  than  under  Low  Uncertainty.  The 
interaction  between  Certainty  and  Experience  was  not  significant  for  situation  awareness  acquisition. 
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Figure  6.  Mean  response  time  (minutes)  for  situation  awareness  (recognition  priming). 


COA  SELECTION 

Figure  7  displays  the  mean  COA  selection  time  results  for  each  group  (High/Low  Experience) 
during  both  the  High-  and  Low-Uncertainty  conditions.  Analysis  of  Variance  revealed  a  significant 
main  effect  for  command-post  experience  (F(l,40)  =  4.27,  g  <  .05).  The  High-Experience  group  took 
less  time  (M  =  3.1  min)  for  response  selection  than  the  Low-Experience  group  (M  =  6.8  min). 

In  addition,  a  significant  main  effect  of  task  certainty  was  found  (F(l,40)  =  4.15,  g  <  .05)  indicating 
that  response  selection  under  High  Uncertainty  took  longer  (M  =  6.85  min)  than  when  available 
information  conveyed  greater  certainty  (M  =  3.2  min).  The  interaction  was  not  statistically  signifi¬ 
cant. 
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UNCERTAINTY 

Figure  7.  Mean  response  time  (minutes)  for  COA  selection. 


COA  EXECUTION 

Figure  8  displays  the  response  execution  results  for  each  group  (High/Lx)w  Experience)  during 
both  High-  and  Low-Uncertainty  conditions.  The  ANOVA  results  indicated  a  significant  main  effect 
for  command-post  experience  (F(l,41)  =  4.55,  p  <  .05).  COA  execution  was  significantly  faster  for 
the  High-Experience  group  (M  =  8.95  min)  than  the  Low-Experience  group  (M  =  12.94  min).  In 
addition,  task  certainty  approached  statistical  significance  (F(l,41)  =  3.55,  p  =  .067)  showing  that 
COA  execution  under  High-Uncertainty  conditions  took  longer  (M  =  12.84  min)  than  under  Low- 
Uncertainty  conditions  (M  =  9.0  min).  The  interaction  between  task  Certainty  and  Experience 
was  significant  (F(l,41)  =  4.84,  p  <  .05).  This  result  is  clearly  displayed  in  figure  8.  The  High- 
Experience  Marines  were  significantly  faster  (M  =  8.4  min)  that  the  Low-Experience  Marines 
(M  =  17.67  min)  to  execute  a  COA  under  the  High-Uncertainty  condition.  Under  the  Low- 
Uncertainty  condition,  the  two  groups  displayed  nearly  identical  execution  times. 
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Figure  8.  Mean  response  time  (minutes)  for  response  execution. 


ACCURACY 

Two  measures  of  task  accuracy  were  employed.  Analyses  were  conducted  to  determine  if  partici¬ 
pants  selected  (1)  the  most  correct  leverage  point  and  (2)  the  most  appropriate  course  of  action 
according  to  the  outcomes  suggested  in  Mastering  Tactics  (Schmitt,  1994). 

Despite  a  relatively  large  difference  in  the  expected  direction,  the  leverage  point  selection 
performances  of  the  two  Experience  groups  were  not  significantly  different.  The  High-Experience 
group  (N  =  23)  leverage  point  accuracy  was  75%,  while  the  Low-Experience  group  (N  =  29) 
achieved  an  accuracy  of  48%. 

In  terms  of  COA  selection,  significantly  more  participants  in  the  High-Experience  group  executed 
responses  in  agreement  with  the  criterion  (87%)  than  the  Low-Experience  group  (48%),  (x\l)  = 
6.52,  p  <  .05). 


RESULTS  OF  SUBJECTIVE  REPORTS 

Analyses  of  self-reported  battle-plan  confidence  or  the  subjective  rating  of  the  effect  of  uncertainty 
resulted  in  no  significant  differences,  either  between  High-  and  Low-Uncertainty  level  or  between 
High-  and  Low-Experience  groups.  In  both  cases,  the  groups  had  similar  scores  that  tended  to  be 
mid-range  on  a  five-point  Likert  scale  for  both  confidence  and  uncertainty. 

Finally,  the  usefulness  of  each  display  tool  was  rated  on  a  5 -point  Likert  scale  (1  =  not  useful, 

5  =  highly  useful).  Table  5  shows  the  average  utility  rating  for  each  of  the  display  tools.  A 
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Kolmogorov-Smimov  test  was  performed  on  the  subjective  ratings  for  each  item  to  determine 
whether  the  distribution  of  scores  departed  significantly  from  a  uniform  or  normal  distribution.  All 
tools  were  rated  as  “useful”  (p  <  .05)  with  the  exception  of  the  Grid  Color  tool,  which  was  rated  as 
“not  useful”  (p  <  .05). 


Table  5.  Average  utility  rating  of  the  display  tools. 


Map  Tool 

utility  in  the  Command 
Post 

Utility  of  Map  Tools  in 
Task 

Battalion  Schematic 

4.15 

3.94 

Grid 

3.48 

3.65 

Grid  Color  Change 

2.17 

1.69 

Message  Archiving 

4.15 

3.96 

Message  Bulletin  Board 

4.37 

4.20 

Message  Post-it 

4.40 

4.33 

Movement  Histories 

4.17 

3.80 

Movement  Indicator 

4.46 

4.33 

Panning 

4.61 

3.74 

Projected  Positions 

4.23 

3.66 

Review  Orders 

4.21 

4.08 

Zooming 

4.33 

3.40 
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DISCUSSION 

The  Recognition-Primed  Decision-Making  model  of  Naturalistic  Decision-Making  (Klein,  1993) 
emphasizes  the  importance  of  investigating  the  dynamics  of  the  decision-making  process  in  an 
applied  environment.  This  experiment  provides  a  unique  example  of  decision-making  in  such  an 
enviromnent.  We  measured  behavior  reflecting  time  to  acquire  situation  awareness,  as  well  as  time 
required  for  (and  accuracy  of)  COA  selection  and  execution. 

SITUATION  AWARENESS 

Our  results  indicated  that  while  experts  take  longer  to  achieve  situation  awareness,  compared  to 
novices,  they  are  significantly  faster  in  choosing  a  course  of  action  and  executing  a  response.  Such 
results  are  consistent  with  the  expert  vs.  novice  literature  (Di  Bello,  1997;  Stokes,  Kemper,  and  Kite, 
1997;  and  Xiao,  Milgram,  and  Doyle,  1997).  Differences  in  the  time  required  to  achieve  situation 
awareness  have  been  explained  by  suggestions  that  experts  have  a  larger  cognitive  database  of 
related  experiences  to  compare  with  the  new  situation,  thus  requiring  more  time.  Furthermore, 
experts  seem  to  know  which  information  is  most  important  and  relevant,  whereas  novices  tend  to 
review  all  available  information  in  a  haphazard  manner  regardless  of  its  actual  relevance  (Lehto, 
1997).  It  can  be  argued  that  the  systematic  review  of  information  by  the  expert  takes  longer  because, 
in  a  dynamic  situation,  the  expert  is  aware  of  the  links  and  contingencies  associated  with  the  data 
under  review  and  consequently  elects  to  search  and/or  wait  for  relevant  updates.  This  result  differs 
from  what  is  found  in  the  static  situation  in  which  the  expert  quickly  realizes  that  no  new  information 
will  be  avdlable  and  that  the  decision  can  be  based  upon  current  data  (St.  John,  Callan,  Proctor,  and 
Holste,  2000).  St.  John  et  al.  showed  that  novices  were  significantly  more  likely  than  experts  to  adopt 
a  “wait  and  see”  approach  in  the  static  situation. 

This  study  also  supports  research  indicating  that  novices  have  a  tendency  to  make  decisions  based 
upon  a  limited  understanding  of  the  situation,  usually  developing  only  a  single  course  of  action 
(Dhar,  1997).  However,  while  one  might  infer  from  earlier  findings  that  novices  would  have  a  ten¬ 
dency  to  change  leverage  points  more  often  than  experts  (due  to  being  more  easily  swayed  by  irrele¬ 
vant  information),  this  was  not  the  case  here.  Our  results  revealed  no  significant  between-group 
differences  in  the  number  of  leverage  points  selected.  Other  studies  have  indicated  that  novices  tend 
to  be  less  capable  of  developing  situation  awareness  than  more  experienced  individuals  due  to  their 
inability  to  distinguish  relevant  from  irrelevant  information.  Although  a  much  larger  proportion  of 
the  High-Experience  Marines  selected  the  correct  tactical  leverage  point  (75%  versus  48%  for  nov¬ 
ices),  this  result  was  not  statistically  significant,  probably  due  to  sample  size.  Several  cells  in  the 
analyses  contained  fewer  than  15  participants,  a  number  often  cited  as  the  suggested  minimum  num¬ 
ber  of  participants  required  for  proper  analyses  (Myers  and  Hansen,  1997).  A  major  problem  in  the 
conduct  of  applied  research,  especially  if  expertise  is  required,  is  the  recruitment  of  a  sufficient  num¬ 
ber  of  qualified  participants  to  obtain  desired  statistical  power. 

COA  SELECTION 

COA  selection  was  faster  in  the  Low-Uncertainty  environment  than  in  the  High-Uncertainty  envi¬ 
ronment.  This  supports  previous  findings  that  response  selection  is  faster  when  information  is  more 
reliable  and  less  ambiguous.  In  addition,  the  High-Experience  group  was  significantly  faster  in 
selecting  a  course  of  action  than  the  Low-Experience  group,  and  this  difference  was  apparent 
regardless  of  the  level  of  information  certainty.  The  High-Experience  group  also  was  significantly 
more  accurate  than  the  Low-Experience  group  in  choosing  the  correct  response.  This  result  is 
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consistent  with  previous  studies  reporting  that  naive  participants  are  limited  in  their  generation  of 
courses  of  action  and  are  highly  influenced  by  irrelevant  information  (e.g.,  Lehto,  1997). 

COA  EXECUTION 

Because  the  experimental  task  was  dynamic — and  the  participants  were  constantly  receiving  new 
information — there  is  a  possibility  that  they  may  have  modified  their  orders  (i.e.,  altered  COA)  sub¬ 
sequent  to  achieving  “situation  awareness”  (which  was  deemed  to  have  occurred  upon  selection  of 
the  Open  Orders  Window).  Therefore,  we  developed  the  measure  of  COA  Execution,  operationally 
defined  as  the  time  between  selection  of  the  final  leverage  point  and  completion  of  the  order.  This 
measure  is  inclusive  of  COA  Selection  and  incorporates  the  time  required  for  the  submission  of  the 
final  order  for  execution. 

The  results  for  COA  Execution  were  very  interesting.  Figure  8  displays  the  differences  in  response 
time  as  a  function  of  Combat  Operations  Center  (COC)  experience  and  degree  of  information 
certainty.  The  High-Experience  group  was  significantly  faster  in  executing  the  selected  course  of 
action  than  the  Low-Experience  group,  but  more  interesting  was  the  significant  interaction  indicating 
that  the  differences  in  response  execution  between  High-  and  Low-Experience  groups  occurred  only 
under  the  High-Uncertainty  condition.  This  suggests  that,  in  a  dynamic  situation,  novices  are 
dramatically  affected  by  ambiguous  or  uncertain  information  (in  terms  of  Response  Execution)  while 
experts  respond  as  quickly  under  either  level  of  certainty. 

These  results  support  the  intuitive  premise  that  decision-making  is  faster  when  information  used  to 
make  the  decision  is  reliable  and  unambiguous.  It  might  be  expected  firom  this  result  that  experts 
would  show  a  greater  confidence  in  their  decisions  and  indicate  that  uncertainty  has  less  of  an  impact 
upon  their  decision-making.  However,  the  data  indicated  that  there  were  no  significant  differences  in 
battle-plan  confidence  as  a  function  of  either  Level  of  Experience  or  Task  Certainty.  Nor  were  there 
any  differences  between  the  High-  and  Low-Experience  groups  regarding  the  subjective  effect  of 
uncertainty  upon  battle-plan  generation.  From  these  data,  it  cannot  be  determined  whether  experts 
have  a  tendency  to  understate  confidence  and  overestimate  the  effect  of  uncertainty,  or  if  it  is  the 
novices  who  tend  to  overstate  confidence  and  underestimate  the  effect  of  uncertainty. 

The  findings  of  this  smdy  clearly  expand  the  novice-expert  decision-making  distinctions  found 
by  St.  John  et  al.  (2(X)0)  in  a  static  task  to  differences  observed  in  a  dynamic  task.  The  results  of  the 
current  study  contrast  with  the  findings  of  St.  John  et  al.  (2000)  in  two  key  areas.  The  current  study 
found  that  (1)  experience  in  the  command  post  affected  time  spent  developing  situation  awareness 
and  (2)  High-Experience  Marines  responded  faster  once  situation  awareness  was  obtained.  The  cur¬ 
rent  task  differed  from  the  previous  study  in  that  it  used  a  dynamic  instead  of  static  display.  Another 
difference  between  studies  was  the  overall  level  of  command-post  experience.  In  the  current  study, 
the  median  of  command-post  experience  was  much  greater  than  reported  by  St.  John  et  al.  (2000) 

(90  versus  14  days,  respectively). 

This  study  also  highlights  an  important  distinction  between  the  ways  experts  tuid  novices  process 
information.  Experts  appear  to  spend  the  majority  of  their  time  assessing  the  situation  to  develop 
situation  awareness.  Once  assessment  is  complete,  they  spend  relatively  little  time  selecting  and 
developing  a  course  of  action  (which  may  be  mentally  linked  to  a  recognized  pattern).  On  the  other 
hand,  novices  spend  less  time  assessing  the  situation,  leading  to  long  delays  in  selecting  and  devel¬ 
oping  a  course  of  action — ^which  may  be  the  result  of  impaired  situation  awareness. 

These  results  clearly  support  the  claim  that  improving  displays  to  enhance  situation  awareness  will 
likewise  enhance  user  performance.  Improved  command-post  displays  should  enhance  the  ability  of 
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the  experienced  Marines  to  provide  a  more  rapid  response  and,  at  the  same  time,  strengthen  the  abil¬ 
ity  of  the  novice  to  acquire  improved  situation  awareness. 

Eleven  of  the  display  tools  used  during  this  task  were  rated  as  useful  for  operational  use  in  the 
command  post.  Only  one  tool  (which  provided  the  ability  to  change  the  color  of  the  grid)  was  judged 
not  to  be  useful.  The  tools  that  were  rated  as  useful  were  incorporated  into  the  prototype  Tactical 
Situation  Awareness  (TacSAT)  display. 

FINAL  NOTE 

This  study  demonstrates  the  importance  of  investigating  decision-making  in  an  applied,  dynamic 
environment.  The  strategy  selected  by  the  decision  maker  (e.g.,  analytic  or  intuitive)  is  highly  task- 
dependent.  When  an  individual  is  provided  ample  time  to  make  a  decision,  an  analytic  approach 
(consistent  with  many  current  training  curricula)  is  frequently  selected.  On  the  other  hand,  in  a 
dynamic  task  in  which  information  uncertainty  is  high  and  a  rapid  decision  is  required,  an  intuitive 
decision-making  strategy  is  usually  chosen.  Therefore,  when  investigating  ways  to  optimize  human 
performance  in  an  applied  setting,  consideration  must  be  given  to  the  context  of  the  task.  Future 
displays  should  be  configured  to  present  information  in  a  way  that  best  supports  the  appropriate 
strategy  (analytic  or  intuitive)  to  optimize  decision-making.  We  have  incorporated  these  findings  and 
ideas  into  the  development  of  the  TacSAT,  an  interface  designed  to  assist  Marine  Corps  COC 
personnel  to  more  rapidly  achieve  situation  awareness  and  develop  one  or  more  suitable  courses  of 
action. 
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